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Abstract.  The  present  study  examined  the  utility  of  testing  noncognitive  aptitudes  as  a  predictor  of  graduation  from  US  Air  Force  air  traffic 
controller  (ATC)  training.  A  total  of  1,003  training  candidates  participated  in  the  study.  Results  of  logistic  regression  indicate  that  noncognitive 
areas  of  functioning  at  the  start  of  training  significantly  predicted  graduation  beyond  the  effects  of  a  measure  of  cognitive  aptitude.  Higher 
levels  of  emotional  self-awareness,  self-actualization,  reality  testing,  stress  tolerance,  happiness,  and  approach  to  problem  solving  best 
differentiated  graduates  from  nongraduates.  The  results  of  the  study  suggest  such  areas  of  functioning  are  important  for  training  success. 
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US  Air  Force  (USAF)  air  traffic  controllers  (ATCs)  are  a 
unique  group  of  military  personnel  in  a  high-risk,  high- 
demand  career  field  responsible  for  monitoring  and  direct¬ 
ing  air  traffic  by  utilizing  radar,  non-radar  capabilities,  and 
various  computer-based  technologies  (USAF,  2006,  2013a). 
They  must  sustain  vigilance,  prioritize,  analyze,  and 
respond  to  a  wide  array  of  visual  and  auditory  information 
in  an  environment  where  information  is  constantly  chang¬ 
ing.  This  includes  evaluating  altitude,  speed,  distance,  and 
direction  of  multiple  aircraft  while  assessing  factors  such 
as  weather  conditions  across  time  zones,  varying  aircraft 
arrival  and  take-off  times,  and  other  operational  issues  to 
ensure  safe  and  effective  management  of  flight  operations. 
The  career  field  is  known  for  a  high  operational  tempo 
where  threats  to  human  safety  and  the  consequences  of  mis¬ 
takes  are  substantial.  The  day-to-day  operational  tasks  of  air 
traffic  control  can  present  many  unpredictable  and  uncon¬ 
trollable  events  in  which  the  effective  management  of  air 
traffic  relies  heavily  upon  the  controller’s  skills  gained  from 
training  and  experience,  as  well  as  his  or  her  inherent  psy¬ 
chological  attributes  (e.g.,  cognitive  aptitude,  stress  toler¬ 
ance,  self-confidence,  approach  to  problem  solving,  etc.). 

Screening  and  Selection  Procedures 

To  ensure  candidates  are  well  suited  for  the  rigors  of  air 
traffic  control,  they  must  meet  stringent  aeromedical  and 


personnel  classification  requirements.  All  training  appli¬ 
cants  must  meet  USAF  “fitness  for  duty”  as  well  as 
ground-based  controller  medical  standards  (USAF, 
2013b).  Applicants  are  disqualified  from  entering  training 
if  they  have  any  current  or  past  history  of  medical  or  psy¬ 
chiatric  illness  reasonably  perceived,  at  the  discretion  of  a 
physician,  to  impair  performance.  If  an  applicant  has  a  his¬ 
tory  of  a  medical  condition  (e.g.,  bacterial  meningitis, 
closed  head  injury,  multiple  sclerosis,  optic  neuritis,  cardio¬ 
vascular  disease,  uncorrected  visual  difficulties)  or  a  psy¬ 
chiatric  illness  (e.g.,  major  depression,  anxiety)  that  may 
affect  functioning  under  the  austere  stresses  of  the  career 
field,  then  he  or  she  is  disqualified  from  entering  training. 
This  is  to  ensure  training  candidates  can  sustain  physical 
and  psychological  health  in  routine  and  emergent  condi¬ 
tions  without  elevating  risks  to  safety  while  performing 
their  duties.  The  medical  screening  standards  and  require¬ 
ments  for  civilian,  Federal  Aviation  Administration  (FAA) 
training  candidates  are  similar  to  USAF  standards  (Office 
of  Personnel  Management,  2014). 

Furthermore,  applicants  are  screened  using  the  Armed 
Services  Vocational  Aptitude  Battery  (ASVAB;  Defense 
Manpower  Data  Center  [DMDC],  2008).  The  ASVAB  is 
a  cognitive  aptitude  test  given  to  select  enlisted  military 
training  applicants.  At  the  time  of  this  study,  the  ASVAB 
consists  of  four  composite  T  scores  (General,  Mechanical, 
Electrical,  and  Administrative).  USAF  ATC  training  appli¬ 
cants  must  obtain  a  minimum  ASVAB  General  composite 
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T-score  of  55  and  a  Mechanical  Composite  T  score  of  55 
(USAF,  2006,  2013a)  before  being  selected  for  training. 
The  ASVAB  General  composite  score  is  derived  from  the 
subtests  of  Arithmetic  Reasoning  and  Verbal  Expression 
(which  is  a  combination  of  scores  from  the  Paragraph  Com¬ 
prehension  and  Word  Knowledge  subscales).  General  cog¬ 
nitive  aptitude  is  a  well-known  predictor  of  performance 
(Schmidt,  2002;  Schmidt  &  Hunter,  1998;  Schmidt  & 
Hunter,  2004),  and  prior  research  has  found  ASVAB  scores 
to  be  predictive  of  training  outcomes  for  ATCs  (Carretta  & 
Frederick,  1999;  Carretta  &  King,  2008;  King,  Manning,  & 
Dreschler,  2006;  McBride,  2012). 


Brief  Description  of  Training  Pipeline 

The  use  of  the  ASVAB  and  required  cut-off  scores  reflects 
the  importance  of  training  candidates  having  a  high  level  of 
cognitive  functioning  to  acquire  the  necessary  skill  sets 
within  a  condensed  period  of  time  (e.g.,  approximately 
1  year).  To  become  fully  certified,  training  candidates  must 
complete  two  phases  of  training  (Manacapilli  et  al.,  2012). 

The  first  phase  is  the  Technical  Training  Course.  This 
course  teaches  the  fundamentals  of  flight,  as  well  as  inter¬ 
national,  national,  and  military  directives  that  govern  air 
operations.  Candidates  are  taught  ATC  phraseology;  how 
to  utilize  maps,  aeronautical  charts,  navigational  aids,  and 
communication  systems;  as  well  as  FAA  rules  and  regula¬ 
tions  governing  the  controlling  of  aircraft.  The  majority 
of  their  training  focuses  on  controlling  aircraft  via  radar 
(which  covers  between  50  and  80  square  miles)  and  tower, 
such  as  the  final  seven  square  miles  related  to  an  aircraft’s 
approach. 

The  second  phase  is  Certification  Training.  Candidates 
who  successfully  pass  technical  training  subsequently  enter 
radar  or  control  tower  operations  for  certification  training. 
This  training  occurs  at  their  first  assigned  operational  duty 
installation  and  is  approximately  9-12  months.  During  cer¬ 
tification  training,  candidates  must  learn  and  demonstrate 
the  ability  to  control  and  regulate  en  route  and  terminal 
air  traffic.  They  must  obtain  qualification  in  initiating  and 
issuing  clearances,  instructions,  and  advisories  to  ensure 
the  safe  and  expeditious  flow  of  air  traffic  operating  under 
instrument  and  visual  flight  rules.  They  must  demonstrate 
skills  in  planning,  organizing,  directing,  and  evaluating  a 
wide  array  of  air  traffic  control  activities. 

The  costs  for  selecting  and  training  USAF  ATC  candi¬ 
dates  are  substantial  and  90%  of  training  failures  are  due 
to  academic  issues  (Manacapilli  et  al.,  2012).  Although 
mistakes  and  performance  errors  are  tolerated  as  training 
candidates  develop  their  skill  sets,  those  who  make 
substantial  and/or  repeated  errors  are  eliminated  from  train¬ 
ing.  The  overall  attrition  rate  is  approximately  49%  and 
occurs  at  a  steady  rate  throughout  training. 

Training  is  a  complex  process  influenced  by  a  range  of 
factors.  Unforeseen  life  events,  medical  illnesses  and  injury, 
major  life  stressors,  the  motivational  level  of  training  candi¬ 
dates,  as  well  as  recruiting  and  training  processes  are  all 
potential  contributors  to  attrition.  While  increasing  the 


number  of  recruits  may  increase  training  production  rates, 
it  is  reasonable  to  consider  a  more  cost-effective  approach 
by  making  improvements  to  the  screening  process 
(Manacapilli  et  al.,  2012). 

Although  current  screening  processes  ensure  candidates 
exemplify  high  levels  of  health  and  cognitive  aptitude,  not 
all  healthy  and  highly  intelligent  candidates  are  successful 
with  adapting  to  rigorous  conditions.  A  logical  step  to 
improving  screening  processes  is  the  addition  of  testing  that 
assesses  noncognitive  functioning  relevant  to  occupational 
training  and  tasks.  There  is  a  growing  body  of  literature 
suggesting  aspects  of  a  person’s  noncognitive  aptitudes 
(i.e.,  emotional,  behavioral,  and  social  functioning)  have  a 
significant  impact  on  job  performance  (Barrick,  2005; 
Barrick,  Mount,  &  Judge,  2001;  Burch  &  Anderson, 
2008;  Day  &  Silverman,  1989;  Judge  &  Hies,  2002;  Le 
et  al.,  2011;  Mount,  Barrick,  &  Stewart,  1998;  Tett  & 
Burnett,  2003;  Tett,  Jackson,  Rothstein,  &  Reddon,  1999; 
Thoresen,  Bradley,  Bliese,  &  Thoresen,  2004).  In  particular, 
noncognitive  aptitudes  (e.g.,  high  levels  of  conscientious¬ 
ness  and  stress  tolerance)  have  been  found  to  distinguish 
civilian  ATCs  from  the  general  population  (Dean,  Russell, 
&  Farmer,  2002;  Karson  &  O’Dell,  1971,  1974; 
King,  Retzlaff,  Detwiler,  Schroeder,  &  Broach,  2003; 
Luuk,  Luuk,  &  Aluoja,  2009;  Nye  &  Collins,  1991;  Schroeder, 
Broach,  &  Young,  1993).  More  recent  studies  of  ATC  training 
candidate  selection  results  found  that  interviewer  ratings  of 
general  motivation,  career-specific  motivation,  and  social 
competence  were  associated  with  subsequent  training  success 
(Conzelmann  &  Keye,  2014;  Pecena  et  al.,  2013).  However, 
the  aforementioned  studies  are  limited  by  correlational 
designs  and  small  samples,  which  restrict  the  ability  to  make 
definitive  conclusions  about  the  predictive  utility  of  noncogni¬ 
tive  functioning.  These  studies  also  do  not  assess  other  areas  of 
noncognitive  functioning  likely  relevant  to  performance  (e.g., 
emotional  self-awareness,  independence,  reality  testing,  and 
self-confidence). 

The  importance  of  noncognitive  functioning  has  been 
perpetuated  by  theories  of  emotional  and  social  intelligence 
(e.g.,  Bar-On,  2004;  Goleman,  1995;  Mayer,  Salovey,  & 
Caruso,  2004).  Such  theories  focus  on  an  array  of  abilities 
(e.g.,  self-regard,  emotional  self-awareness,  independence, 
self-actualization,  stress  tolerance,  flexibility,  optimism, 
interpersonal  demeanor)  not  currently  addressed  by 
traditional  tests  or  theories  of  intelligence  and  cognitive 
functioning.  One  test  measuring  these  abilities  is  the  Emo¬ 
tional  Quotient  Inventory  (EQ-i).  Higher  scores  on  the  EQ-i 
scales  are  associated  with  better  coping  in  stressful  situa¬ 
tions  (Slaski  &  Cartwright,  2003)  and  higher  levels  of  phys¬ 
ical  and  emotional  health  (Day,  Therrien,  &  Carroll,  2005). 
Although  others  have  found  the  test  to  be  susceptible  to  fak¬ 
ing  (Day  &  Carroll,  2008;  Grubb  &  McDaniel,  2007),  the 
test  has  positive  and  negative  impression  management 
scales  to  assess  an  individual’s  response  style  to  test  items. 
Although  there  is  criticism  regarding  the  accuracy  of  mea¬ 
suring  noncognitive  competencies  via  self-report  (Cooper 
&  Petrides,  2010),  the  EQ-i  continues  to  be  applied  in  a 
variety  of  academic  and  real-world  settings  (Lievens, 
Klehe,  &  Libbrecht,  2011)  with  acceptable  psychometric 
properties,  such  as  internal  consistency,  convergent  validity, 
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and  resistance  to  response  style  and  bias  (Dawda  &  Hart, 
2000;  Kun  et  al,  2012). 

However,  theories  of  emotional  and  social  intelligence 
are  controversial.  Matthews,  Zeidner,  and  Roberts  (2002) 
noted  such  theories  are  often  vague.  Others  have  argued 
labeling  noncognitive  abilities  as  a  form  of  intelligence  is 
a  misinterpretation  of  the  intelligence  construct  (Locke, 
2005).  The  most  prominent  criticism  of  emotional  and 
social  intelligence  theories  is  centered  on  the  results  of  stud¬ 
ies  that  have  found  the  domains  and  facets  of  such  con¬ 
structs  to  resemble  (and  are  highly  correlated  with) 
personality  traits  and  taxonomies  (Austin,  2009;  Fiori  & 
Antonakis,  2011;  Landy,  2005;  Mikolajczak,  Luminet, 
Leroy,  &  Roy,  2007;  Newsome,  Day,  &  Catano,  2000; 
Petrides  &  Fumham,  2001;  Petrides,  Pita,  &  Kokkinaki, 
2007;  Siegling,  Saklofske,  Vesely,  &  Nordstokke,  2012; 
Smith,  Ciarrochi,  &  Heaven,  2008).  The  self-report  items, 
factor  structure,  item  content,  and  response  choices  have 
overlapping  content  and  structure  with  commonly  used, 
standardized  personality  tests.  For  example,  many  of  the 
factors  in  the  Bar-On  measure  of  emotional  intelligence 
are  closely  related  to  personality  traits  measured  by  the 
16  Personality  Factor  Questionnaire  (16PF  5th  edition; 
Cattell,  Cattell,  &  Cattell,  1999),  the  Occupational  Personal¬ 
ity  Scale  (Fumam,  Race,  &  Rosen,  2014),  and  the  NEO  Per¬ 
sonality  Inventory  (PI)-3  (McCrae,  2000).  As  a  result,  the 
content  and  factor  structure  of  theories  and  tests  of  emotional 
and  social  intelligence  are  likely  a  mixed  model  of  noncog¬ 
nitive  functioning  that  overlaps  with  personality  traits. 

Regardless  of  the  criticism  and  debate  over  the  con¬ 
structs  of  emotional  and  social  intelligence  versus  theories 
of  personality,  anecdotal  discussion  the  authors  of  this  study 
had  with  USAF  ATC  leadership  reveal  a  consensus  between 
training  and  career  field  managers,  that  candidates  who  suc¬ 
cessfully  adapt  to  training  are  perceived  to  have  higher  lev¬ 
els  of  assertiveness,  self-regard,  emotional  self-awareness, 
and  independence,  as  well  as  adaptability,  stress  tolerance, 
and  positive  mood,  whether  considered  a  personality  trait 
or  form  of  emotional/social  intelligence  (Bar-On,  2004). 
However,  the  assessment  of  noncognitive  functioning  is 
not  accounted  for  in  current  USAF  screening  processes. 


Purpose  of  Study 

The  purpose  of  this  exploratory  study  is  to  evaluate  pre¬ 
training,  standardized  noncognitive  aptitude  testing  (in  con¬ 
junction  with  cognitive  testing)  from  USAF  air  traffic 
control  candidates  in  order  to  identify  (a)  cognitive  and 
noncognitive  areas  of  functioning  that  distinguish  those 
who  pass  versus  fail  training,  as  well  as  (b)  the  incremental 
validity  of  noncognitive  aptitude  testing.  On  the  basis  of  the 
studies  cited  in  the  previous  sections,  we  hypothesized 
graduates  would  have  higher  levels  of  intrapersonal  and 
interpersonal  functioning,  adaptability,  stress  management, 
and  general  mood  and  the  collective  combination  of  cogni¬ 
tive  and  noncognitive  areas  of  functioning  would  improve 
the  predictive  validity  of  selection  process. 


Method 

Participants 

Data  were  collected  from  1,003  training  candidates 
( n  =  808,  80.6%  male;  n  =  195,  19.4%  female)  over  the 
course  of  2  years.  Mean  age  for  male  and  female 
participants  was  20.84  ( SD  =  2.63)  years  and  20.77 
( SD  =  2.24)  years,  respectively.  Race  and  ethnicity  were 
not  assessed.  A  total  of  512  (51.0%)  completed  all  training 
requirements  and  graduated,  and  a  total  of  491  (49.0%)  did 
not  graduate. 


Measures 

Armed  Services  Vocational  Aptitude  Battery  (ASVAB) 

The  ASVAB  (DMDC,  2008)  was  used  to  assess  cognitive 
aptitude.  Four  composite  scores  are  based  on  weighted 
combinations  of  seven  subtests  (see  Table  1  for  descrip¬ 
tions).  The  ASVAB  composite  scores  have  good  reliability, 
correlate  with  academic  achievement,  and  are  predictive  of 
subsequent  military  performance  (Campbell  &  Knapp, 
2001;  Welsh,  Kucinkas,  &  Curran,  1990).  Composite  scores 
reflect  the  respondent’s  percentile  scores  relative  to  the 
normative  population,  whereas  subtest  scores  are  standard¬ 
ized  with  a  mean  score  of  50  and  a  standard  deviation  (SD) 
of  10. 


Emotional  Quotient  Inventory  (EO-i) 

The  EQ-i  (Bar-On,  2004)  was  used  to  assess  noncognitive 
areas  of  functioning.  The  test  was  chosen  because  the  con¬ 
structs  of  the  test  appear  to  measure  areas  of  noncognitive 
functioning  relevant  to  ATC  training.  The  test  is  composed 
of  133  items  that  load  onto  five  major  domain  scales  and  15 
subscales  (see  Table  2  for  descriptions).  Items  are  rated  on  a 
5-point  Likert  scale  ranging  from  very  seldom  true  or  not 
true  of  me  to  very  often  true  of  me  or  true  of  me.  Scale 
scores  are  normed  for  the  general  population,  with  a  stan¬ 
dardized  mean  score  of  100  and  SD  of  15.  Internal  consis¬ 
tency  estimates  for  the  EQ-i  subtests  are  >  .76,  with  test- 
retest  reliability  estimates  of  .85  after  1  month  and  .75  after 
4  months  (Bar-On,  1997,  2000).  The  reliability  estimate  for 
this  sample  was  .94. 


Training  Outcome 

Training  outcome  was  measured  as  a  binary  variable  and 
candidates  were  identified  as  either  graduates  or  nongradu¬ 
ates.  Participants  who  voluntarily  dropped  out  of  training 
(i.e.,  “self-eliminated”)  and/or  were  not  allowed  to  proceed 
further  owing  to  insufficient  performance  were  categorized 
in  the  nongraduate  group.  Participants  who  were  removed 
from  training  for  medical  reasons  were  not  included  in  this 
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Table  1.  Composite  scale  and  subtest  descriptions  of  the  ASVAB 


Subtests/composite  scales 

Subtests 

General  Science 
Arithmetic  Reasoning 
Mathematical  Knowledge 
Electronics  Information 
Mechanical  Comprehension 
Verbal  Expression 

Auto  and  Shop  Information 
Composite  Scales 
General 
Mechanical 
Electrical 

Administrative 


Descriptions 


Knowledge  of  physical  and  biological  sciences 
Ability  to  solve  arithmetic  word  problems 
Knowledge  of  high  school  mathematics  principles 
Knowledge  of  electricity  and  electrons 
Knowledge  of  mechanical  and  physical  principles 

Ability  to  select  the  correct  meaning  of  words  presented  in  a  context,  to  identify 
the  best  synonym  for  a  given  word,  and  to  obtain  information  from  written  passages 
Knowledge  of  automobile  technology  and  shop  terminology  and  practices 

Combination  of  Verbal  Expression  and  Arithmetic  Reasoning 

Combination  of  Mechanical  Comprehension,  General  Science,  and  Auto  &  Shop  Information 
Combination  of  Arithmetic  Reasoning,  Mathematical  Knowledge,  Electronics  Information,  and 
General  Science 
Verbal  Expression 


Note.  ASVAB  =  Armed  Service  Vocational  Assessment  Battery. 


Table  2.  Subscale  descriptions  of  the  EQ-i 


Domains/ subscales 
Intrapersonal 

Emotional  Self-Awareness 
Assertiveness 

Sell-Regard 

Self-Actualization 

Independence 

Interpersonal 

Empathy 

Interpersonal  Relationship 

Social  Responsibility 

Adaptability 

Problem  Solving 

Reality  Testing 

Flexibility 

Stress  Management 
Stress  Tolerance 

Impulse  Control 

General  Mood 
Happiness 
Optimism 

Validity  Scale 

Positive  Impression 
Inconsistency 


Subscale  descriptions 


The  ability  to  recognize  one’s  feelings  and  emotions 

The  ability  to  express  feelings,  beliefs,  and  thoughts  and  to  defend  one’s  right  in  a 
nondestructive  manner 

The  ability  to  respect  and  accept  oneself  as  basically  good 

The  ability  to  realize  and  strive  to  achieve  one’s  potential  capacities 

The  ability  to  be  self-directed  and  self-controlled  in  one's  thinking  and  actions 
and  to  be  free  of  emotional  dependency 

The  ability  to  be  aware  of,  to  understand,  and  to  appreciate  the  feelings  of  others 

The  ability  to  establish  and  maintain  mutually  satisfying  relationships  that  are  characterized 
by  intimacy  and  by  giving  and  receiving  affection 

The  ability  to  demonstrate  oneself  as  a  cooperative,  contributing,  and  constructive  member 
of  one’s  social  group 

The  ability  to  identify  and  define  problems  as  well  as  to  generate  and  implement  potentially 
effective  solutions 

The  ability  to  assess  the  correspondence  between  what  is  experienced  and 
what  objectively  exists 

The  ability  to  adjust  one’s  emotions,  thoughts,  and  behavior  to  changing  situations  and 
conditions 

The  ability  to  withstand  adverse  events  and  stressful  situations  without  “falling  apart” 
by  actively  and  positively  coping  with  stress 

The  ability  to  resist  or  delay  an  impulse,  drive,  or  temptation  to  act 


The  ability  to  feel  satisfied  with  one's  life,  to  enjoy  oneself  and  others,  and  to  have  fun 
The  ability  to  look  at  the  brighter  side  of  life  and  to  maintain  a  positive 
attitude,  even  in  the  face  of  adversity 

The  tendency  to  present  oneself  in  an  overly  favorable  or  unrealistically  positive  manner 
The  tendency  to  respond  to  items  in  an  inconsistent,  random,  or  unreliable  manner 


Note.  EQ-i  =  Emotional  Quotient  Inventory. 
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study.  However,  only  a  small  portion  (less  than  6%)  fail  to 
complete  training  due  to  medical  issues  (Manacapilli  et  al., 
2012). 


Procedures 

ASVAB  testing  was  completed  prior  to  the  start  of  basic 
military  training  as  a  routine  part  of  USAF  entrance  screen¬ 
ing  processes.  The  EQ-i  was  administered  to  candidates 
upon  enlistment  in  the  USAF  and  prior  to  entering  ATC 
training.  Candidates  were  informed  that  the  test  would  not 
be  used  in  the  selection  process.  Upon  completion  of  the 
EQ-i,  participants’  tests  were  scored  and  matched  with  their 
ASVAB  scores.  Subsequent  training  outcomes  (graduate  vs. 
nongraduate)  were  then  merged  with  testing  scores.  The 
final  database  was  then  de-identified  for  the  purposes  of 
data  analyses. 


Data  Analysis 

The  focus  of  this  study  was  to  determine  the  incremental 
validity  of  a  noncognitive  measure  as  value  added  when 
used  in  addition  to  cognitive  measures  in  assessing  out¬ 
comes  of  ATC  trainees  from  the  course  of  initial  entry  into 
training.  All  statistical  analyses  were  performed  using 
SAS®  software,  Version  9.3.  This  study  focused  on  overall 
graduation  rates  from  both  the  course  of  initial  entry  and 
the  upgrade  certification  training.  While  the  graduation  rate 
from  the  course  of  initial  entry  was  77%  (734/1,003),  the 
overall  graduation  rate  after  upgrade  certification  training 
was  51%  (512/1,003).  This  is  crucial  in  that  only  those 
trainees  who  successfully  completed  upgrade  certification 
training  in  either  tower  or  radar  controlling  actually  become 
operational. 

Range  restriction  procedures  were  considered  for  both 
measures.  Since  there  are  minimum  ASVAB  score  require¬ 
ments  to  enter  ATC  training,  the  ASVAB  scores  were  range 
restricted.  Multivariate  range  restriction  correlation  adjust¬ 
ments  were  calculated  (Lawley,  1943),  and  subsequent 
logistic  regression  analyses  were  conducted  on  uncorrected 
correlations  (see  Table  3).  However,  range  restriction  with 
noncognitive  aptitude  testing  is  less  clear.  Since  such  testing 
was  not  used  for  selection  and  there  are  no  minimum  or 
maximum  requirements  for  test  scores,  direct  range  restric¬ 
tion  did  not  appear  to  apply  to  such  a  measure.  While  indi¬ 
rect  range  restriction  is  possible  with  the  correlated  EQ-i 
and  ASVAB  measures,  correlations  are  small  to  moderate, 
and  therefore  corrections  for  range  restriction  were  not  per¬ 
formed  for  the  EQ-i. 

Group  means  and  standard  deviations  for  all  ASVAB 
and  EQ-i  variables  used  in  the  study  are  presented  in 
Table  3.  Hierarchal  logistic  regression  was  utilized  to  assess 
the  impact  of  noncognitive  aptitude  testing  scores  on  the 
ability  to  predict  pass/fail  training  outcomes.  All  cognitive 
and  noncognitive  aptitude  subtests  were  considered  regard¬ 
less  of  their  individual  impact  on  prediction.  Using  the  stan- 
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dard  threshold  of  10  records  per  predictor  (Hosmer  & 
Lemeshow,  2000),  all  22  potential  predictors  were  consid¬ 
ered  for  inclusion  in  the  model.  Initial  logistic  regressions 
were  run  for  ASVAB  only,  EQ-i  only,  and  combined  models 
(see  Table  4).  This  was  followed  by  hierarchal  stepwise 
regressions  to  limit  the  model  to  only  those  variables  that 
demonstrated  specific  predictive  validity  (see  Table  4). 
ASVAB  composites  and  EQ-i  domain  scores  are  linear 
combinations  of  subtest  scores,  therefore  they  were  not  used 
in  model  building  in  combination  with  subtest  scores. 
Goodness  of  fit  for  each  model  was  determined  by  the 
increase  in  R2  values  as  a  result  of  including  scores  from 
the  EQ-i,  as  well  as  the  results  of  receiver  operating  charac¬ 
teristic  (ROC),  Hosmer-Lemeshow,  and  Akaike  informa¬ 
tion  criterion  (AIC)  analyses.  Further,  standardized  beta 
estimates  were  used  to  establish  the  potential  impact  of  each 
variables  contribution  to  the  model. 


Results 

Between-groups  t  tests  were  performed  on  both  the  Positive 
Impression  Scale  and  Inconsistency  Index.  There  were  no 
differences  between  groups  on  the  Positive  Impression  scale 
t  (1,002)  =  —0.21,/)  =  .83,  which  suggests  that  graduates’ 
higher  scores  on  the  EQ-i  were  not  due  to  a  greater  ten¬ 
dency  to  present  themselves  in  a  more  favorable  positive 
manner.  Nongraduates  did  score  higher  on  the  Inconsis¬ 
tency  Index,  t  (1,002)  =  2.74,  p  =  .01,  suggesting  they 
may  have  been  slightly  less  careful  or  reliable  in  respond¬ 
ing.  However,  mean  scores  on  the  Inconsistency  Index 
( M  =  4.46  for  nongraduates,  M  =  3.99  for  graduates)  were 
well  below  12,  the  recommended  threshold  for  the  test  to  be 
considered  invalid. 

In  the  first  step  of  logistic  regression  procedures,  all 
ASVAB  subtest  scores  were  entered.  The  resulting  model 
was  statistically  significant  and  accounted  for  approxi¬ 
mately  8.3%  of  training  outcomes,  Wald  %2  (7)  =  58.49, 
p  <  .001,  R2  =  0.083.  The  Hosmer-Lemeshow  statistic 
was  .07  suggesting  the  model  may  not  be  a  good  fit,  with 
ROC  =  .64  and  AIC  =  1,325.98.  In  the  second  step,  all 
15  EQ-i  subscale  scores  were  assessed,  which  yielded  a  sta¬ 
tistically  significant  model  that  accounted  for  approxi¬ 
mately  8.7%  of  training  outcomes,  Wald  %2  (15)  =  60.28, 
p  <  .001,  R2  =  .087.  The  Hosmer-Lemeshow  statistic  was 
.52  suggesting  the  model  may  be  a  good  fit,  with 
ROC  =  .65  and  AIC  =  1,322.69,  both  suggesting  a  moder¬ 
ate  fit.  When  both  ASVAB  and  EQ-i  predictors  were  com¬ 
bined  into  one  compensatory  model,  the  model  remained 
significant,  Wald  /2  (9)  =  106.25,  p  <  .001,  R2  =  .16). 
The  Hosmer-Lemeshow  statistic  was  .94  suggesting  the 
model  may  be  the  best  fit,  with  ROC  =  .70  and 
AIC  =  1262.50.  The  reduction  in  AIC  to  1,262.50  suggests 
that  the  combined  model  is  a  better  fit  than  either  of  the 
instruments  alone  (see  Table  4). 

The  second  step  was  to  perform  hierarchal  stepwise 
regression  to  potentially  identify  the  specific  predictors  of 
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Table  3.  ASVAB  and  EQ-i  descriptive  statistics  and  correlations  with  graduate/nongraduate  outcomes  ( N  = 

1,003) 

Subtest  and  composite  scales 

Nongraduate 

Graduate 

ASVAB  Subtests 

M  (SD) 

M  (SD) 

r 

rc 

General  Science 

56.34  (5.95) 

57.71  (5.90) 

.12** 

.34 

Arithmetic  Reasoning 

57.63  (5.56) 

59.59  (5.28) 

.18** 

.38 

Mathematical  Knowledge 

57.47  (5.54) 

59.61  (4.99) 

20** 

.40 

Electronics  Information 

55.57  (6.79) 

56.94  (7.18) 

.10** 

.29 

Mechanical  Comprehension 

57.96  (6.39) 

59.40  (6.38) 

1 1** 

.32 

Verbal  Expression 

56.69  (4.67) 

57.43  (4.56) 

.08** 

.27 

Auto  and  Shop  Information 
ASVAB  Composites 

51.40  (6.96) 

53.36  (7.56) 

.13** 

.24 

General 

76.26  (12.47) 

80.43  (11.63) 

Mechanical 

73.76  (13.18) 

78.81  (12.99) 

Electrical 

76.21  (13.33) 

81.46  (11.97) 

Administrative 

75.28  (12.93) 

80.13  (11.81) 

EQ-i  Composites  and  Subscales 
Intrapersonal 

102.48  (14.60) 

107.38  (13.15) 

Emotional  Self-Awareness 

104.26  (13.91) 

107.02  (12.92) 

.10** 

Assertiveness 

104.45  (14.54) 

108.84  (13.97) 

15** 

Self-Regard 

103.04  (13.72) 

107.31  (11.71) 

17** 

Self-Actualization 

100.75  (14.59) 

104.55  (12.92) 

.14** 

Independence 

97.17  (15.16) 

101.88  (13.48) 

.16** 

Interpersonal 

102.41  (14.65) 

105.13  (13.62) 

Empathy 

101.66  (14.89) 

103.65  (15.34) 

.07* 

Interpersonal  Relationship 

102.83  (14.51) 

105.61  (13.74) 

.10** 

Social  Responsibility 

101.07  (14.27) 

103.36  (13.28) 

.08** 

Adaptability 

101.60  (14.28) 

108.43  (13.19) 

Problem  Solving 

100.02  (14.25) 

102.86  (13.44) 

.10** 

Reality  Testing 

99.67  (15.04) 

105.40  (13.76) 

.20** 

Flexibility 

104.42  (15.19) 

109.19  (13.29) 

17** 

Stress  Management 

102.57  (15.03) 

106.96  (13.63) 

Stress  Tolerance 

104.64  (15.20) 

111.24  (13.20) 

23** 

Impulse  Control 

99.82  (14.61) 

103.48  (13.84) 

.13** 

General  Mood 

103.07  (14.16) 

108.12  (12.01) 

Happiness 

104.58  (13.39) 

108.55  (11.22) 

.16** 

Optimism 

101.50  (15.13) 

106.85  (12.97) 

19** 

Validity  Index 

Positive  Impression 

110.07  (15.01) 

110.27  (15.09) 

Inconsistency  Index 

4.46  (2.71) 

3.99  (2.72) 

Notes.  Column  headings  indicate  observed  correlations  (V)  and  correlations  corrected  for  range  restriction  ( rc ).  Statistical  significance 
was  tested  only  for  the  observed  correlations.  ASVAB  =  Armed  Service  Vocational  Assessment  Battery.  EQ-i  =  Emotional  Quotient 
Inventory.  *p  <  .05.  **p  <  .01. 


training  success  (see  the  variables  in  italics  in  Table  4).  The 
resulting  model  was  statistically  significant  and  accounted 
for  approximately  16%  of  training  outcomes,  Wald  %2 
(9)  =  124.30, <  .001,  R2  =  .16,  ROC  =  .70,  and  the  Hos- 
mer-Lemeshow  (Hosmer  &  Lemeshow,  2000)  probability 
of  .95,  suggests  a  good  model  fit.  In  addition,  the  AIC  sta¬ 
tistic  was  minimal,  as  for  the  full  model  AIC  =  1,265.71. 
Although  the  model  does  not  meet  all  criteria  at  the  highest 
level,  it  is  identified  as  a  good  fit  to  the  data. 

Table  5  presents  cross-validation  results.  The  established 
leave-one-out  cross-validation  classification  method  is  a 
procedure  whereby,  in  these  data,  each  trainee  is  classified 
based  on  the  model  developed  from  all  other  trainees  over 
1,003  iterations.  The  guideline  is  that  the  overall  one-out 
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cross-validation  results  should  be  within  10%  of  the  origi¬ 
nal  classification,  which  is  the  case  in  this  study.  While 
the  leave-one-out  method  is  widely  accepted,  it  is  not 
without  its  limitations.  Therefore,  it  is  recommended  that 
a  A'- fold  cross-validation  be  performed  whereby  10-30  ran¬ 
dom  test  samples  are  created  and  classified  based  upon  the 
trained  model  of  the  remaining  data  (Ounpraseuth,  Lensing, 
Spencer,  &  Kodell,  2012).  The  result  of  the  average 
Mold  =  10  classification  is  reported  in  Table  5.  The  classi¬ 
fication  cut-off  criteria  was  set  at  .40.  This  cut-off  was  con¬ 
sidered  the  appropriate  criterion  to  minimize  false-negative 
and  false-positive  error  rates  while  maximizing  correct 
classification  rates.  Current  screening  procedures  result  in 
approximately  five  out  of  10  candidates  graduating,  while 
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Table  4.  ASVAB  and  EQ-i  logistic  regression  model  predicting  ATC  training  outcome  (graduate  vs. 

nongraduate) 

Predictor 

Wald  x2 

P 

STB 

MODEL  STB* 

ASVAB  Subtests 

General  Science 

0.19 

0.66 

2.0 

Arithmetic  Reasoning* 

1.92 

0.17 

6.6 

7.0 

Mathematical  Knowledge* 

16.68 

<.0001 

19.7 

19.2 

Electronics  Information 

0.24 

0.62 

2.4 

Mechanical  Comprehension 

0.59 

0.44 

3.4 

Verbal  Expression 

0.57 

0.45 

3.2 

Auto  and  Shop  Information* 

13. 71 

0.00 

17.4 

16.0 

EQ-i  Subscales 

Emotional  Self-Awareness* 

0.59 

0.44 

4.6 

7.4 

Assertiveness 

0.03 

0.86 

1.0 

Self-Regard 

0.01 

0.92 

0.7 

Self-Actualization  * 

1.02 

0.31 

6.4 

6.7 

Independence 

0.36 

0.55 

3.1 

Empathy 

0.03 

0.86 

1.2 

Interpersonal  Relationship 

0.74 

0.39 

6.5 

Social  Responsibility 

0.06 

0.81 

1.2 

Problem  Solving* 

2.58 

0.11 

8.4 

6.5 

Reality  Testing* 

4.34 

0.04 

14.0 

18.5 

Flexibility 

0.13 

0.72 

2.1 

Stress  Tolerance* 

6.51 

0.01 

17.8 

19.2 

Impulse  Control 

0.73 

0.39 

4.3 

Happiness* 

2.02 

0.16 

10.1 

11.0 

Optimism 

1.20 

0.27 

8.1 

Note.  Significant  predictors  included  in  the  combined  model  are  indicated  with  an  asterisk  and  are  in  italicized  font.  ASVAB  =  Armed 
Service  Vocational  Assessment  Battery.  EQ-i  =  Emotional  Quotient  Inventory. 


Table  5.  Leave-one-out  cross-validation  classification  of  observed  outcomes  compared  with  predicted  outcomes  for  the 
Combined  Model 


Predictions  using  both  ASVAB  and  EQ-i  with  .40  cut-off  criteria 

Passed 

Failed 

Total 

Predicted  to  Pass 

436 

Sensitivity  85% 

Of  those  who  passed  you  are  correct 

85  out  of  100  times 

193 

False  positive  51% 

Of  those  who  failed  you  are  wrong  5 1 
out  of  100  times 

629 

Predicted  to  Fail 

76 

False  negative  26% 

Of  those  who  passed  you  are  wrong  26  out  of 
100  times 

298 

Specificity  61% 

Of  those  who  failed  you  are  correct  61 
out  of  100  times 

374 

Current  total 

512  (51%) 

491  (49%) 

1,003 

Notes.  ASVAB  = 

Armed  Service  Vocational  Assessment  Battery.  EQ-i  = 

Emotional  Quotient  Inventory. 

the  proposed  model  predicts  seven  out  of  10  candidates 
graduating  (i.e.,  20%  improvement  in  production). 


Discussion 

The  current  exploratory  study  sought  to  determine  whether 
the  assessment  of  noncognitive  areas  of  functioning  among 
ATC  candidates  prior  to  the  start  of  training  predicts 
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subsequent  graduation.  Results  were  consistent  with  our 
hypothesis  that  graduates  would  (a)  have  higher  levels  of 
intrapersonal  and  interpersonal  functioning,  adaptability, 
stress  management,  and  general  mood  and  that  (b)  the 
inclusion  of  noncognitive  aptitude  testing  with  cognitive 
testing  would  improve  prediction  of  training  outcomes. 

ATC  graduates  scored  significantly  higher  on  all  ASVAB 
subtests.  However,  based  on  the  results  from  the  logistic 
regression,  three  ASVAB  subtests  (i.e.,  Mathematical 
Knowledge,  Arithmetic  Reasoning,  and  Auto  and  Shop 
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Information)  significantly  predicted  graduation  outcomes. 
These  findings  suggest  that,  in  terms  of  cognitive  aptitude, 
greater  knowledge  of  high-school-level  mathematic  princi¬ 
ples  and  increased  familiarity  with  mechanical  technology 
and  practices  may  be  especially  important  for  success  in 
the  ATC  career  field.  These  findings  align  with  those  from 
an  earlier  study  by  Carretta  and  King  (2008),  who  also  found 
the  ASVAB  to  be  a  strong  predictor  of  training  outcomes. 
The  findings  are  also  consistent  with  previous  research  by 
McBride  (2012),  who  found  that  successful  graduation  from 
the  US  Marine  Corps’  ATC  training  program  was  predicted 
by  performance  on  the  ASVAB  composite  scales  of  Mathe¬ 
matical  Knowledge  and  Auto  and  Shop  Information  subtests. 
Mathematical  aptitude,  along  with  other  cognitive  abilities, 
has  similarly  been  identified  as  a  significant  predictor  of  suc¬ 
cessful  task  performance  among  civilian  ATC  personnel 
(Ackerman  &  Kanfer,  1993).  As  a  result,  these  three  ASVAB 
subtests  appear  to  be  important  areas  of  functioning  that  are 
relevant  to  ATC  career  field  training  outcomes.  Mathematics, 
for  example,  is  an  essential  skill  set  for  ATC  personnel,  who 
are  required  to  calculate  and  consider  factors  such  as  velocity, 
distance,  altitude,  and  spatial  relationships  when  directing 
aircraft.  Regardless  of  the  highly  computerized  and  auto¬ 
mated  processes  that  are  available  to  assist  controllers,  math¬ 
ematical  aptitude  continues  to  remain  a  significant  predictor. 

In  terms  of  noncognitive  functioning,  ATC  graduates 
had  higher  mean  scores  on  all  EQ-i  composites  and  scales 
indicating  higher  levels  of  noncognitive  functioning  across 
the  domains  of  interpersonal  and  intrapersonal  functioning, 
stress  management,  adaptability,  and  general  mood.  Com¬ 
parable  findings  for  the  16PF  have  been  reported  among 
nonmilitary  ATC  personnel  (Karson  &  O’Dell,  1974). 
The  results  of  this  study  also  suggest  that  the  inner  drive 
to  objectively  validate  one’s  internal  experiences  with  exter¬ 
nal  reality  (i.e.,  reality  testing)  and  to  effectively  and  con¬ 
structively  recognize  and  manage  one’s  emotions  (i.e., 
emotional  self-awareness)  as  well  as  to  remain  emotionally 
calm  and  sustain  a  positive  emotional  demeanor  (i.e.,  stress 
tolerance  and  happiness)  under  demanding  conditions  are 
especially  important  characteristics  of  successful  ATC  can¬ 
didates.  This  finding  converges  with  previous  findings  that 
civil  ATC  personnel  report  greater  emotional  stability  than 
the  general  population  (Luuk  et  al.,  2009). 

The  relative  contribution  of  emotional  self-awareness, 
self-actualization,  reality  testing,  stress  tolerance,  happiness, 
and  approach  to  problem  solving  in  the  prediction  of  ATC 
graduation  relative  to  the  other  areas  of  noncognitive  apti¬ 
tude  may  be  due  to  their  mutual  relationship.  For  example, 
the  ability  to  recognize,  self-monitor,  and  regulate  a  positive 
inner  emotional  state  of  functioning  refers  to  the  amount  of 
attention  an  individual  directs  toward  his  or  her  internal 
states  and  behaviors,  and  the  extent  to  which  these  behaviors 
align  with  one’s  goals  (Kanfer  &  Ackerman,  1989).  ATC 
candidates  who  possess  the  ability  to  accurately  judge  their 
internal  states  and  behaviors  with  respect  to  external  objec¬ 
tive  indicators  of  performance  and  to  actively  respond  to 
stress  and  difficult  conditions  in  an  effective  manner  (i.e., 
problem  solving)  may  experience  greater  self-efficacy, 
which  has  been  found  to  be  correlated  with  performance 
among  ATC  personnel  (Ackerman  &  Kanfer,  1993). 


Greater  reality  testing,  stress  tolerance,  happiness,  and 
an  inner  drive  to  achieve  one’s  fullest  potential  (i.e.,  self- 
actualization)  may  also  contribute  to  better  training  out¬ 
comes  because  ATC  candidates  with  higher  levels  of  these 
characteristics  may  perceive  they  have  the  ability  to  compe¬ 
tently  manage  and  effectively  influence  and/or  control  situa¬ 
tions  (i.e.,  problem  solving).  Flowever,  additional  research  is 
needed  to  determine  how  such  areas  of  functioning  fully 
influence  job  performance  and  interact  over  time.  Regard¬ 
less,  the  results  of  the  study  suggest  that  the  inclusion  of  non¬ 
cognitive  functioning  has  added  value  to  current  screening 
procedures  aimed  at  reducing  training  attrition. 

There  are  both  similarities  and  differences  in  training 
and  job  demands  between  USAF  and  civilian  FAA  air  traf¬ 
fic  control  training  (Air  Force  Enlisted  Classification 
Directory,  2013;  FAA,  2013;  US  Department  of  Labor, 
2014).  Both  agencies  have  a  tiered  process  that  begins  with 
classroom  and  simulation  training  where  candidates  are 
taught  using  lecture,  computer-based  instruction,  and  simu¬ 
lation  with  a  range  of  fidelity  and  events  that  progressively 
mimic  on-the-job  knowledge  and  skill  requirements.  This 
initial  phase  lays  the  foundation  of  common,  fundamental 
air  traffic  control  procedures.  The  next  tier  of  training 
moves  to  on-the-job  training  within  an  operational  facility 
where  trainees  work  with  live  traffic  and  are  provided  with 
close  oversight  and  supervision.  However,  while  in  training, 
USAF  ATC  trainees  must  also  complete  professional  mili¬ 
tary  education,  duties,  and  events  outside  of  their  career 
field  that  are  part  of  their  USAF  enlistment  requirements. 
These  are  not  requirements  among  nonmilitary  ATC  train¬ 
ing  candidates.  As  a  result,  USAF  ATC  candidates  may 
have  additional  occupational  stressors  that  increase  the  rig¬ 
ors  of  training.  Although  there  are  similarities,  there  are 
likely  enough  differences  between  civilian  and  military  pro¬ 
grams  that  caution  is  recommended  when  generalizing  the 
findings  of  this  study. 

Study  Limitations 

Conclusions  based  on  the  current  study  should  be  made 
cautiously  owing  to  certain  limitations.  The  current  study 
utilized  a  self-report  methodology,  which  could  be  vulnera¬ 
ble  to  response  bias,  especially  within  the  context  of  person¬ 
nel  selection.  This  limitation  may  account  for  the  fact  that 
in  the  current  sample,  scores  on  the  Positive  Impression 
Index  of  the  EQi  were  approximately  one  standard  devia¬ 
tion  above  the  normative  mean,  suggesting  a  tendency  for 
participants  to  present  themselves  in  a  desirable  manner. 
Additional  studies  should  be  conducted  to  determine  the 
impact  of  response  bias  on  the  predictive  validity  of  self- 
report  measures  prior  to  widespread  inclusion  of  noncogni¬ 
tive  aptitude  testing  among  screening  and  selection  proce¬ 
dures.  Furthermore,  evaluating  the  contribution  of  two 
instruments  on  training  outcomes  using  a  combined  model 
that  accounts  for  range  restriction  in  one  instrument 
(ASVAB,  cognitive  ability  test)  and  not  the  other  (EQ-I, 
noncognitive  aptitude  test)  may  give  a  biased  impression 
of  the  importance  of  one  set  of  predictors  in  relation  to 
the  other.  In  this  case,  it  is  perceived  the  statistical  model 
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may  result  in  biased  interpretations  of  the  uncorrected 
range-restricted  predictors  as  having  more  or  perhaps  less 
importance  than  the  unrestricted  predictors.  The  authors 
recognize  this  concern  and  acknowledge  that  there  may 
be  a  certain  unintended  bias  in  predictor  interpretation 
and  recommend  further  study  is  needed  to  address  these 
issues.  Another  limitation  is  related  to  the  characteristics 
of  the  sample,  which  was  composed  of  military  personnel 
from  a  single  branch  of  service.  The  USAF  population  at 
large  is  an  aggregate  of  many  subgroups  and  therefore 
results  among  subgroups  cannot  be  generalized  to  one 
another  easily.  It  is  also  possible  that  study  results  do  not 
generalize  to  other  ATC  samples.  However,  the  fact  that 
USAF  ATC  candidates  must  pass  all  FAA  requirements 
for  certification  and  graduation  lends  some  confidence  to 
the  external  validity  of  these  findings.  Despite  these  limita¬ 
tions,  the  current  study  provides  information  about  areas  of 
noncognitive  functioning  that  may  contribute  to  training 
success. 
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